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Abstract. It is shown that from two strings that are partially random 
and independent (in the sense of Kolmogorov complexity) it is possible 
to effectively construct polynomially many strings that are random and 
pairwise independent. If the two initial strings are random, then the 
above task can be performed in polynomial time. It is also possible to 
construct in polynomial time a random string, from two strings that have 
constant randomness rate. 
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1 Introduction 

This paper belongs to a line of research that investigates whether certain at- 
tributes of randomness can be improved effectively. We focus on finite binary 
strings and we regard randomness from the point of view of Kolmogorov com- 
plexity. Thus, the amount of randomness in a binary string x is given by K{x), 
the Kolmogorov complexity of x and the randomness rate of x is defined as 
K{x)/\x\, where |a;| is the length of x. Roughly speaking, a string x is considered 
to be random if its randomness rate is approximately equal to 1. It is obvious that 
randomness cannot be created from nothing (e.g., from the empty string). On the 
other hand, it might be possible that if we already possess some randomness, we 
can produce "better" randomness or "new" randomness. For the case when we 
start with one string x, it is known that there exists no computable function that 
produces another string y with higher randomness rate (i.e., "better" random- 
ness), and it is also clear that there is no computable function that produces 
"new" randomness, by which we mean a string y that has non-constant Kol- 
mogorov complexity conditioned by x. In fact, Vereshchagin and Vyugin |VV02[ 
Th. 4] construct a string x with high Kolmogorov complexity so that any shorter 
string that has small Kolmogorov complexity conditioned by x (in particular any 
string effectively constructed from x) has small Kolmogorov complexity uncon- 
ditionally. Therefore, we need to analyze what is achievable if we start with two 
or more strings that have a certain amount of randomness and a certain degree 
of independence. In this case, in certain circumstances, positive solutions exist. 
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For example, Fortnow, Hitchcock, Pavan, Vinodchandran and Wang [FHP+06] 
show that, for any cr there exists a constant i and a polynomial-time procedure 
that from an input consisting of i n-bit strings xi , . . . , , each with Kolmogorov 
complexity at least crn, constructs an n-bit string with Kolmogorov complexity 
^ n — dep(xi, . . . , xi) (dep(a;i, . . . , xi) measures the dependency of the input 
strings and is defined as X)i=i K{xi) — K{xi . . . Xi); cz means that the inequality 
holds within an error of O(logn)). 

In this paper we focus on the case when the input consists of two strings 
X and y of length n. We say that x and y have dependency at most a{n) if 
the complexity of each string does not decrease by more than a{n) when it is 
conditioned by the other string, i.e., if K{x) — K{x \ y) < a{n) and K{y) — K{y \ 
x) < a{n). The reader should have in mind the situation a(n) = O(logn), in 
which case we say that x and y are independent (see [CZ08| for a discussion of 
independence for finite binary strings and infinite binary sequences) . We address 
the following two questions: 

Question 1. Given x and y with a certain amount of randomness and a certain 
degree of independence, is it possible to effectively /efficiently construct a string 
z that is random? 

Question 2. (a more ambitious version of Question 1) Given x and y with 
a certain amount of randomness and a certain degree of independence, is it 
possible to effectively /efficiently construct strings that are random and have 
small dependency with a;, with y, and pairwise among themselves? How many 
such strings exhibiting "new" randomness can be produced? 

A construction is effective if it can be done by a computable function, and it 
is efficient if it can be done by a polynomial-time computable function. 

We first recall the well-known (and easy-to-prove) fact that if x and y are 
random and independent, then the string z obtained by bit- wise XOR-ing the 
bits of X and y is random and independent with x and with y. Our first result 
is an extension of the above fact. 

Theorem 1. (Informal statement.) If x and y are random and have dependency 
at most a{n), then by doing simple arithmetic operations in the field GF[2"] 
(which take polynomial time), it is possible to produce polynomially many strings 
zi, . . . , Zpoiy(„) of length n such that K(zi) > n — a{n) and the strings x, y, 
zi, . . . , Zpoiy(„) are pairwise at most ~ Q;(n)-dependent, where ~ (^) means that 
the equality (resp., the inequality) is within an error of O(logn). In particular, if 
X and y are independent, then the output strings are random and together with 
the input strings form a collection of pairwise independent strings. 

The problem is more complicated when the two input strings x and y have 
randomness rate significantly smaller than 1. In this case, our questions are 
related to randomness extractors, which have been studied extensively in com- 
putational complexity. A randomness extractor is a polynomial-time computable 
procedure that improves the quality of a defective source of randomness. A source 
of randomness is modeled by a distribution X on {0, 1}", for some n, and its 
defectiveness is modeled by the min-entropy of X {X has min-entropy k if 2~'^ 



is the largest probability that X assigns to any string in {0, 1}"). There are sev- 
eral type of extractors; for us, multi-source extractors are of particular interest. 
An ^-multisource extractor takes as input £ defective independent distributions 
on the set of n-bit strings and outputs a string whose induced distribution is 
statistically close to the uniform distribution. The analogy between random- 
ness extractors and our questions is quite direct: The number of sources of the 
extractor corresponds to the number of input strings and the min-entropy of 
the sources corresponds to the Kolmogorov complexity of the input strings. For 
£ = 2, the best multisource extractors are (a) the extractor given by Raz [RazOSj 
with one source having min-entropy ((1/2) -I- o)n (for some small a) and the 
second source having min-entropy polylog(n), and (b) the extractor given by 
Bourgain |Bou05] with both sources having min-entropy ((1/2) — a)n (for some 
small a). Both these extractors are based on recent results in arithmetic com- 
binatorics. It appears that finding polynomial-time constructions achieving the 
goals in Question 2 is difficult. If we settle for effective constructions, then pos- 
itive solutions exist. In [Zi m09j . we have shown that there exists a computable 
function / such that if x and y have Kolmogorov complexity s{n) and depen- 
dency at most a(n), then f{x,y) outputs a string z of length m w s{n)/2 such 
that K{z \ x) >z m — a{n) and K{z | y) ^ ?7i — a{n). Our second result extends 
the methods from |Zim09j and shows that it is possible to effectively construct 
polynomially many strings exhibiting "new" randomness. 

Theorem 2. (Informal statement.) For every function O(logn) < s{n) < n, 
there exists a computable function / such that if x and y have Kolmogorov 
complexity s{n) and dependency at most a(n), then /(x, y) outputs polynomially 
many strings zi, . . . , 2;poiy(n) of length ra « s(n)/3 such that K{zi) > m — a{n) 
and the strings (x, y, Zi, . . . , Zpaiy(^n)) pairwise at most w Q!(n)-dependent. In 
particular, if x and y are independent, then the output strings are random and 
together with the input strings form a collection of pairwise independent strings. 



For Question 1, we give a polynomial-time construction in case x and y have 
linear Kolmogorov complexity, i.e., K{x) > 6n and K{y) > Sn, for a positive 
constant (5 > 0. The proof relies heavily on a recent result of Rao [Rao08| . which 
shows the existence of 2-source condensers. (A 2-source condenser is similar but 
weaker than a 2-source extractor in that the condenser's output is only required 
to be statistically close to a distribution that has larger min-entropy rate than 
that of its inputs, while the extractor's output is required to be statistically close 
to the uniform distribution.) 

Theorem 3. (Informal statement.) For every constant (5 > 0, there exists a 
polynomial-time computable function / such that if x and y have Kolmogorov 
complexity Sn and dependency at most a{n), then f{x,y) outputs a string z of 
length m — fi{5n) and K{z) > m — {a{n) + poly(logn)). 

The main proof technique is an extension of the method used in |Zim08) 
and in |Zim09| . It uses ideas from Fortnow et al. FHP+06] , who showed that a 



multi-source extractor can also be used to extract Kolmogorov complexity. A key 
element is the use of balanced tables, which are combinatorial objects similar to 



2-source extractors. A balanced table is an A^-by-A^ table whose cells are colored 
with M colors in such a way that each sufficiently large rectangle inside the table 
is colored in a balanced way, in the sense that all colors appear approximately 
the same number of times. The exact requirements for the balancing property 
are tailored according to their application. The type of balanced table required in 
Theorem 2 is shown to exist using the probabilistic method and then constructed 
using exhaustive search. This is why the transformation in Theorem 2 is only 
effective, and not polynomial-time computable. The existence of the type of 
balanced table used in Theorem 3 is a direct consequence of Rao's 2-source 
condenser. 

The paper is structured as follows. Sections ll.ll and [L2l introduce the notation 
and the main concepts of Kolmogorov complexity. Section 11.31 is dedicated to 
balanced tables. Theorem 1 and Theorem 2 are proved in Section[21 and Theorem 
3 is proved in Section [31 

1.1 Preliminaries 

N denotes the set of natural numbers. For n € N, [n] denotes the set {1, 2, . . . , n}. 
We work over the binary alphabet {0, 1}. A string is an element of {0, 1}*. If x 
is a string, \x\ denotes its length. The cardinality of a finite set A is denoted \ A\. 
Let M be a standard Turing machine. For any string x, define the Kolmogorov 
complexity of x with respect to M, as Km{x) = min{|p| | M{p) = x}. There is a 
universal Turing machine U such that for every machine M there is a constant 
c such that for all x, Ku{x) < Km{x) -f c. We fix such a universal machine 
U and dropping the subscript, we let K(x) denote the Kolmogorov complexity 
of X with respect to U . For the concept of conditional Komogorov complexity, 
the underlying machine is a Turing machine that in addition to the read/work 
tape which in the initial state contains the input p, has a second tape containing 
initially a string y, which is called the conditioning information. Given such a 
machine M, we define the Kolmogorov complexity of x conditioned by y with 
respect to M as Km{x \ y) = min{|p| | M{p,y) — x}. Similarly to the above, 
there exist universal machines of this type and a constant c and they satisfy the 
relation similar to the one above, but for conditional complexity. We fix such a 
universal machine U , and dropping the subscript C/, we let K{x \ y) denote the 
Kolmogorov complexity of x conditioned by y with respect to U . In this paper, 
the constants implied in the O(-) notation depend only on the universal machine. 

The Symmetry of Information Theorem (see ^ZLTOj) states that for all strings 
X and y: 

\[K{x) - K{x I y)) ~ {K{y) ~ K{y \ x))\ < Oi\ogK{x) + \ogK{y)). (1) 

In case the strings x and y have length n, it can be shown that 

\iK{x) - K{x I y)) - {K{y) - K{y \ x))\ < 21ogn + 0(1). (2) 

Sometimes we need to concatenate two strings a and 6 in a self-delimiting 
matter, i.e., in a way that allows to retrieve each one of them. A simple way to do 



this is by taking 01010202 . . . o„o„01b, where o = oi . . . o„, with each € {0, 1}. 
A more efficient encoding is as follows. Let |o| in binary notation be C1C2 . . . Cfc. 
Note that k = [log |a|J + f . Then we define concat(o, b) = C1C1C2C2 . . . CkCkOlab. 
Note that |concat(a, 6)| = \a\ + \b\ + 2[log |o|J + 4. 

1.2 Independent strings 

Definition 1. (a) Two strings x and y are at most a{n)- dependent if K{x) — 

K{x\y) < a{\x\) and K{y) - K{y\x) < a(\y\). 

(b) The strings {xi, X2, ■ ■ ■) are pairwise at most a{n)- dependent, if for every 
i i; o-i^<i o-f^ o,t most a{n)- dependent. 

1.3 Balanced tables 

A table is a function T : \N] x [A^] [Af]. In our applications, A^ and M are 
powers of 2, i.e., A" = 2" and M = 2™. We identify [A^] with {0, 1}" and [M] 
with {0, 1}™. Henceforth, we assume this setting. 

It is convenient to view such a function as a two dimensional table with A' 
rows and A' columns where each entry has a color from the set [M] . If .Bi , B2 
are subsets of [A^] , the Bi x B2 rectangle of table T is the part of T comprised 
of the rows in Bi and the colmims in i?2. If ^ C {0, 1}™ and {x, y) e [A'] x [A'], 
we say that the cell {x, y) is ^-colored if T{x, y) G A. 

In our proofs, we need the various tables to be balanced, which, roughly 
speaking, requires that in each sufficiently large rectangle Bi x B2, all colors 
appear approximately the same number of times. 

One variant of this concept is given in the following definition. 

Definition 2. Let k G N. The table T is {S,n'^)- strongly balanced if for every 
pair of sets Bi and B2, where Bi C [AT], > S, B2 C [AT], IB2I > S, the 
following two inequalities hold: 

(1) For every a e [M], 

\{{x,y) e Bi X B2 I T{x,y) = a}\ < ^\B, x B2I, 

(2) for every (a, 6) e [M]^ and for every G [n*']^, 

\{{x,y) e Bi X S2 I Tix + i,y) = a andT{x + j,y) = b}\ < -^|Bi x B2I, 

where addition is done modulo N. 

Using the probabilistic method, we show that, under some settings for the pa- 
rameters, strongly-balanced tables exist. 



Lemma 1. If > 3M^\nM + 6M^-k-lnn + 6SM'^ + 6SM^ + 6SM^ln{N/S) + 
3M^, then there exists an {S,n'') - strongly balanced table. 



NOTE: The condition is satisfied if M = o{{l/ ^)S^/^). 



Proof. We first fix (a, 6) € [Mf , two sets Bi and B2 with BiQ[N],\Bi\= S, 
B2 Q [N], \B2\ = S. Note that for a fixed cell (x, y) e BiX B2 and fixed j e [n% 
Prob[T(a;, y)^a] = l/M and Prob[T(x, y) = a and T{x + j, y) = b] = 1/AP. 
Therefore, by the ChernofF bounds, 



Prob 



number of a-colored cells in Bi x B2 1 

> 2— 



52 M 

and, for fixed j, 
Prob 



< e 



-(l/3)(l/M)S^ 



number of (a, 6)-colored j-apart cells in Bi x B2 1 



5*2 M2 



< e 



-(l/3)(l/Af^)S^ 



There are M possibilities for choosing a, and the number of possibilities for 
choosing the sets Bi and B2 is (^)^ < (eN/S)^'^ ^ p2S+2Sin{N/s) ^ Therefore, 
the probability that the relation (1) in Definition [2] does not hold is bounded by 

g-(l/3)(l/M)S^+lnM+2S+2Sln(Ar/S) 

There are M'^ possibilities for choosing (a, 6), n^'' possibilities for and the 
number of possibilities for choosing the sets Bi and B2 is (J^) < (eN/S)"^^ = 
^2S+2Sin{N/s) ^ Therefore, the probability that the relation (2) in Definition [2] 
does not hold is bounded by 



e 



~(l/3)(l/M^)S^+21n Af+2fclnn+25'+2Sln(Ar/S) 



(4) 



If the parameters satisfy the requirement stated in the hypothesis, then the 
bound in Equation ^ is less than e^^ < 1/2 and the bound in Equation ([4]) 
is less than e"^ < 1/2. Therefore the probability that both relation (1) and 
relation (2) hold is positive, and thus there exists a {S,n^)- strongly balanced 
table. I 

The above proof uses the probabilistic method which does not indicate an 
efficient way to construct such tables. In our application, we will build such 
tables by exhaustive search, an operation that can be done in EXPSPACE. 

A weaker type of a balanced table can be constructed in polynomial-time 
using a recent result of Rao [RaoOSj . We first recall the following definitions. Let 
X and Y be two probability distributions on {0, 1}". The distributions X and 
Y are e-close if for every A C {0, 1}", |Prob(X G A) - Prob(y e A)\ < e. The 
min-entropy of distribution X is maXag{o,i}"(log(l/Pi'ot)(X = a))). 

Fact 1 IRaoOSf For every 6 > 0, e > 0, there exists a constant c and a 
polynomial-time computable function Ext : {0, 1}" x {0, 1}" {0, 1}™, where 
m = Q{5n), such that if X and Y are two independent random variables 
taking values in {0, 1}" and following distributions over {0, 1}" with min- 
entropy at least 5n, then Ext{X, Y) is e-close to a distribution with min-entropy 
m— {5\og\/eY . 



Rao's result easily implies the existence of a polynomial-time table with a 
useful balancing property. 

Lemma 2. Let S > 0, e > Q and let c be the constant and Ext : {0, 1}" x 
{0,1}" — > {0,1}™ be the function from Theorem]^ corresponding to these pa- 
rameters. We identify {0, 1}" with [N] and {0, 1}™ with [M] and view Ext as 
an [N] X [N] table colored with M colors. Then for every rectangle Bi x B2 ^ 
[N] X [N], where \Bi\ > 2^" and jSal > 2^" and for every A C [M], the number 
of cells in Bi x B2 that are A-colored is at most 

Proof Let Bi and B2 be two subsets of {0,1}" of size > 2*". Let X and Y 
be two independent random variables that follow the uniform distributions on 
Bi, respectively B2 and assume the value on {0, 1}" — Bi, respectively on 
{0, 1}" - B2. Since X and Y have min-entropy > 2^'\ it follows that Ext{X, Y) 
is e-close to a distribution Z on {0, 1}™ that has min-entropy m — (fJlogl/e)"^. 
If A C {0,1}™, then Z assigns to A probability mass at most M2(*'°ei/<^)", 
because it assigns to each element in {0,1}™ at most 2"(™~('^''°s i/e)") Thus, 
Ext{X,Y) assigns to A probability mass at most 1^2^^^°^^^"^" + e. This means 
that the number of occurrences of ^-colored cells in the Bi x B2 rectangle is 

bounded by f ^^2^^^°si^/<^y + e) ■ \Bi x B2\. | 



2 Generating multiple random independent strings 

We prove Theorem 1. The formal statement is as follows. 

Theorem 1. For every k G N, there is a polynomial-time computable function f 
that on input xi,X2, two strings of length n, outputs strings x^, X4, . . . , x^k_^_2, 
strings of length n, with the following property. For every sufficiently large n and 
for every function a (n) , if Xi and X2 satisfy 

(i) K{xi) > n — logn, 

(a) K{x2) > n — logn, and 

(Hi) Xi and X2 are at most a{n)- dependent, 

then 

(a) K{xi) > n — {a{n) + {k + 0(1)) logn, for every i G {3, . . . ,n'' + 2}, and 

(b) the strings xi, X2, . . . , a;„fc+2 ^.^^ pairwise at most a{n) -f (3fc + 0(l)) log n- 
dependent. 

Proof. Let xi, X2 € {0, 1}" be such that K{xi) > n — logn, K{x2) > n — logn. 
Since xi and X2 are at most a(n)-dependent, K(xi \ X2) > K{xi) — a(n) > 
n — (a(n) -|- logn). Similarly, K(x2 \ Xi) > n — (a(n) -|- logn). 



The function / outputs 



X3 = Xi + 1- X2, 
Xi = Xi + 2- X2, 

Xnk+2 = Xi+n'' ■ X2, 

where the arithmetic is done in the finite field GF[2"] and 1,2, ... ,n*^ denote 
the first (in some canonical ordering) n'^ non-zero elements of GF[2"]. 

Let Xi be one of the "new" strings, i.e., i G {3, . . . , n*^ + 2}. Let t be defined 
by K{xi I xi) = t. Observe that given xi, i (that can be described with fclogn 
bits) and t + 0{1) bits we can construct X2; first we compute Xi and then from 
Xi and Xi, we derive X2- 

Therefore, K{x2 | a;i) < t + fclogn + 2(logfc + loglogn) + 0(1). Since K{x2 \ 
xi) > n — {a(n) + logn), it follows that t > n — (a{n) + (k + 0(1)) logn) (taking 
into account that k < n; if k > n, the theorem holds trivially). Therefore, 
K{xi I Xi) >n — {a{n) + {k + 0(1))) logn (which implies (a)). We infer that 

K{xi) - K{xi I xi) <{n + 0(1)) - (n - (a(n) + {k + 0(1)) logn)) 
= a(n) + (A; + 0(l))logn. 

By the Symmetry of Information Theorem, K{xi) — K{xi \ Xi) < a{n) + {k + 
0(1)) logn, and thus Xi and Xi arc at most a{n) + {k + 0{1)) logn-dependent. 

Similarly, Xi and X2 are at most Q!(n) + (A; + 0(1)) log n-dependent. Thus, (b) 
follows for pairs {xi,xi) and {xi,X2) with i > 3. 

Let us next consider a pair of strings {xi, xj) with j and i,j G {3, . . . , n*^-!- 
2}. Let t be defined by K{xi \ Xj) = t. Note that given Xj, i and j and t + 0(1) 
bits we can construct Xi : first we compute a;, and then from Xi and Xj , we deive 
xi. Therefore, 

K{xi I Xj) < i + 2fc logn + 2(logfc + log logn) + 0(1). 

Recall that 

K{xi)-K{xi \xj) < a{n) + {k + 0{l))\ogn. 

Then, 

f + 2fclogn + 2(logfc + loglogn) + 0(1) > K{xi \ Xj) 

> kIxi) - (a(n) + {k + 0(1)) logn) 

> n - (a(n) + {k + 0(l))logn). 

Thus, K{xj \xi)=t>n- (a(n) + (3fc + 0(1)) logn). It follows that 

K{xj) - K{xj I Xi) < (n + 0(1)) - (n - {a{n) + {3k + 0(1)) logn)) 
< a(n) + (3A; + 0(1)) logn. 

Thus, Xj and Xi are at most a{n) + {3k + 0(1)) logn-dependent. | 



We next prove Theorem 2. The formal statement is as follows. 



Theorem 2. For every fc G N, for every computable function s(n) verifying 
(6fc + 15) log n < s{n) < n for every n, there exists a computable function f 
that, for every n, on input two strings Xi and X2 of length n, outputs strings 
of length m = s(n)/3— (2A:+5) logn with the following property. 
For every sufficiently large n and for every function a{n), if 

(i) K{xi) > s{n), 

(a) K{x2) > s{n) and 

(Hi) X\ and xi are at most a{n) - dependent, 
then 

(a) K{xi) > m — {a{n) + O(logn)), for every i G {3, . . . , n*^ + 2} and 

(b) the strings in the set {xi,X2, ■ ■ ■ ,a;„fc_|_2} are pairwise at most a{n) + (2k+ 
O ( 1 ) ) log n - dependent. 

Proof We fix n and let iV = 2", to = s{n)/3 - {2k + 5) logn, M = 2", 5 = 
22s(n)/3_ -y^g g^jg^ ^-g^j^g ^ _ q,(|^) _|_ 7 logn. The requirements of Lemma [1] are 

satisfied and therefore there exists a table T : [N] x [N] [M] that is (S", n'^)- 
strongly balanced. By brute force, we find the smallest (in some canonical sense) 
such table T. Note that the table T can be described with logn + 0(1) bits. 
The function / outputs 

X3 = T{xi + 1,X2), 

X4: — T{xi + 2, X2), 

a;„fc+2 = T{xi +n'',X2). 
We show the following two claims. 

Claim 1 For every j € {3, . . . ,n'' + 2}, K{xj \ xi) > K{xj) - (a(n) + 0(log n)) 
and K{xj \ X2) > K{xj) ~ (a(n) + O(logn)). 

Claim 2 For every i,j e {3, . . . , n'' + 2}, K{xj I x^) > K{xj) - (a(n) + {2k + 
0(l)))logn. 

Claim [T] is using ideas from the paper [Zim09j . For the sake of making this 
paper self-contained we present the proof. Let j S {3, . . . , n*^ + 2}. We show 
that K{xj I xi) and K{xj \ X2) are at least to — a{n) — 71ogn. We show this 
relation for K{xj \ x^) (the proof for K{xj \ x\) is similar). Suppose that 
K{xj I X2) < m — a{n) — 71ogn = m — t. Let ti — K{xi). Note that ti > s{n). 
Let B = {u e {0, 1}" I K{u) < ti}. Note that 2*1+^ > \B\ > 22''(")/3 = S. {B 
has size > 22''(")/3 because it contains the set 0"(")/3{o, l}2«(")/3.)We say that a 
column u € [N] is bad for color a G [M] and B if the number of occurrences of 
a in the B x {it} subrectangle of T is greater that (2/M) • \B\ and we say that u 
is bad for B if it is bad for some color a and B. For every a G [M], the number 
of u's that are bad for a and B is < S (since T is (5*, n'^)-strongly balanced and 
we can take into account the first balancing property of such tables). Therefore, 



the number of u's that are bad for B is < M ■ S. Given ti and a description of 
the table T, one can enumerate the set of u's that are bad for B. This imphes 
that any u that is bad for B can be described by its rank in this enumeration 
and the information needed to perform the enumeration. Therefore, if u is bad 
for B, 

K{u) < log(Af • S) + 2(logti + logn) + 0(1) 

< TO + 2s(n)/3 + 41ogn + 0(1) 

< s{n), 

provided n is large enough. Since K{x2) > s{n), it follows that X2 is good for B. 

Let A = {w e[M]\ K{w \x2) <m- t}. We have \A\ < 2""* and, by our 
assumption, xj E A. Let G be the subset of B of positions in the strip B x {0:2} 
of T having a color from A (formally, G = proji(T"^(A) n {B x {X2})) ■ Note 
that Xi is in G. Each color a occurs in the strip B x {X2} at most (2/Af) • \B\ 
(because X2 is good for B). Therefore the size of G is bounded by 

1^1 • (2/M) • \B\ < 2"-* • (2/Af) • 2*1+1 < 2*i-*+2. 

Given X2,ti,m — t and a description of the table T, one can enumerate the set 
G. Therefore, xi can be described by its rank in this enumeration and by the 
information needed to perform the enumeration. It follows that 

K{xi I X2) < ti-t + 2 + 2{logti+log{fn-t)+\ogn) + 0{l) 
< ti -t + 61ogn + 0(l) 
= ti~ a{n) - log n + 0(1) 
= K{xi) - a{n) - logn + 0(1), 

which contradicts that xi and X2 have dependency at most a(n). 
We next prove Claim [2l 

We fix two elements i ^ j in {3, . . . , fc + 2} and analyze K(xi\xj). 

Let ti = K{xi) and ^2 — K{x2)- From hypothesis, ti > s(n) and t2 > s(n). 
We define Bi = {u € {0, 1}" | K{u) < h} and B2 ^ {u G {0, 1}" | K{u) < tz}- 
We have S < \Bi\ < 2*1+^ and S < IB2I < 2*^+1. {Bi and B2 have size larger 
than S = 22^(")/3^ because they contain the set 0^(")/3{0, l}2^(»)/3.) 

Let T~^{xi,Xj) denote the set of pairs (m, u) G [N] x [iV] such that T{u + 
i, w) = a;i and T{u + j, w) ~ Xj. 

Note that (xi,a;2) G T^^{xi,Xj) n (i?i x _B2). Since the table T is strongly 
balanced, 

\T~J{x,,Xj) n (Bi X 52)1 < ^2*i+*^+2 = 2*i+*^-2-+3. 

Note that T;^/ (x, , a;^ ) n {Bi x B2) can be effectively enumerated given Xi, Xj, z, 
j, and the table T. Thus a;ia;2 can be described from xtXj, the rank of {xi,X2) 
in the above enumeration, z, j, and the table T. This implies that 

K{xiX2) < <i + i2 - 2to + 3 + K{xiXj) + 2/clogn + 2(logfc + log log 7i) + O(logn) 
< ti + ^2 - 2to + K{xiXj) + {2k + 0(1)) log n. 



On the other hand, K{xiX2) > K{xi) + K{x2 \ xi) — O(logn) and K{x2 \ xi) > 
K{x2) ~ a{n). Therefore, 

K{xiX2) > K{xi) + K{x2) - {a{n) + O(logn)) 
= ti+t2- {a{n)+0{\ogn)). 

Combining the last two inequahties, we get that 

ti+t2- ia{n) + 0(log n)) < h + t2 - 2ni + K{xiXj) + {2k + 0(1)) log n 

which implies that 

K{x,Xj) > 2m-a{n) - (2fc + 0(1)) log n. 

Therefore 

K{xj\xi) > K{xiXj) — K{xi) — Oilogn) 

> (2m - a{n) - {2k + 0(1)) log n) - (m + 0(1)) - 0(log n) 
= m - a{n) - {2k + 0(1)) log n. 

It follows that 

K{xj) - K{xj I Xi) < (m + 0(1)) - (m - a{n) - {2k + 0(1)) log n) - O(logn) 
< a{n) + (2fc + 0(l))logn. 

Thus, Xj and Xi are at most a{n) + {2k + 0(1)) logn)-dependent. | 

3 Polynomial-time generation of one random string 

In this section we prove Theorem 3. The formal statement is as follows. 

Theorem 3. For every 5 > Q and for every function a{n), there exists a con- 
stant c and a polynomial-time computable function f : {0, 1}" x {0, 1}" — > 
{0, 1}"\ where m = f2{Sn), with the following property. If n is sufficiently large 
and X and y are two strings of length n satisfying 

(i) K{x) > 6n 

(ii) K{y) > Sn 

(Hi) X and y are at most a{n) - dependent, 
then 

K{f{x, y))>m^ {a{n) + 0((logn)^)). 

Proof. Let e = l/(8n^'^ • a{n)) and let c be the constant and Ext : {0,1}" x 
{0, 1}" {0, l}™ be the function given by Theorem [T] for parameters ((5/2) and 
e. Let t = a{n) + lOlogn + (((5/2) log l/e)= + 3 = a(n) + 0((log n)'=). 

The function / on input x and y returns z = Ext{x,y). We show that 
K{z) >m-t. 

Suppose K{z) < m — t. 

Let ti = K{x), t2 = K{y), Bi = {u e {0,1}" | K{u) < ii}, B2 = {u G 
{0, 1}" I K{u) < t2}. From hypothesis, ti > 5n and t2 > 6n. 



Note also that 2*"/^ < \Bi\ < 2*i+i and 2'^"/2 < j^^j < 2*2+1 (rpj^g g^^g 
and B2 have size > 2'^"/2 because they contain 0"-''"/2{0, l}*"/^ ) 
Let 

A = {v e {0, 1}™ I K{v) <m-t} 

We focus on the table defined by the function Ext : [N] x [N] — > [M] , where, as 
usual, we have identified {0, 1}" with [N] and {0, 1}" with [M]. 

Let G be the subset of Bi x B2 of cells in the rectangle Bi x B2 that are 
A-colored. 

Since Ext{x,y) = z E A, x E Bi and y E B2, the cell {x,y) belongs to the 
rectangle Bi x B2 and is A-colored. In other words, x E G. 

Taking into account Lemma [21 we can bound the size of G by 

(^N.2(W2)iogiAr + ^^|5^ x^^l 

<- 2*i+t2+2 ^ 2^'J' . 2(('5/2) log 1/0" + 2" '°S 1/^^ 

_ 2*i+*2-t+(((5/2) log l/c)'^+2 _|_ 2tl+*2-log l/£+2 
< 2*i+*2-(a(«) + 101ogn) 

The last inequality follows from the choice of e and t. 

The set G can be enumerated if we are given ti, t2, 5 and n (from which 
we can derive t and the table Ext), and every element in G can be described 
by its rank in the enumeration and by the information neeeded to perform the 
enumeration. 

Since x € G, it follows that 

K{xy) < ti+t2~a{n) ~ 10 logn + 2(logti + logt2 + logn) + 0(1) 
< ti +t2 — a{n) — 4 log n 
= K{x) + K{y) — a{n) — 4 log n. 

We have used the fact that ti < n + 0(1) and t2 < n + 0(2). By the Symmetry 
of Information Theorem, 

K{xy) > K{y)+K{x \ y) - 2 log n- 0(1). 

Combining the last two inequalities, we get 

K{x) ~ K{x I y) > a{n) + logn, 

which contradicts the fact that x and y are at most a(n)-dependent. | 
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